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NETWORKING SYSTEMS 



Th: prcscnl invenlion relates lo neiworking systems and the forwarding and routing of information therein, 
being mnre particularly directed lo the problems of a common method for managing both cell and packet or frame 
switching in ihc same device, having common hardw.ire. common QoS (Quality of Service) algorithms, common 
forwarding algorithms: building a switch that handles frame switching without interfering with cell switching. 

Background oF Invention 

Two architectures driving networking solutions are cell switching and frame forwarding. Cell switching 
involve.^ the iran.';mission of data-in fixed sire units called ccll.s. This is based on technology refcned to as 
Asynchronous Transfer Mode (ATM). Frame forwarding transmits data in arbitrary size units referred to either as 
frames or packets. The basis of frame forwarding is used by a variety of protocols, the most noteworthy being the 
Internet Protocol (IP) suite. 

The present invention is concerned with forwarding cells and frames in a common system utilizing common 
forwarding algorithms. In co-pending U.S. patent application Serial No. 581.467. filed December 29.1995. for High 
Performance Universal Mulii-Poricd Inicmally Cached Dynamic Random Accejs Memory System. Architecture and 
Method, md co-pending U.S. patent application Serial No. 900.757. filed July 25. 1997. for System Architecture for 
and Method of Dual Path D.nta Processing and Man-igement of Packets and/or Cells and the Like, both of common 
assignee herewith, a promising solution of common cell/frame forwarding is provided. 

Most iradiiionni Intcrncl-.^tylc hosi-io-host data communication h carried out in variable size packet format, 
interconnected by networks (defined as a collection of switches) using packet switches called routers. Recently. 
ATM has become widely avail.nble as a technology to move data between hosts, having been developed to provide a 
common method for .^ending Irarfitinnal telephony dat.i as well as d.ita for computer-to-computcr communication. 
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T>,c P.vi„.. .cHo. c.plo,.d ... ,0 .pp„ Dlv,.on K,.,.ip,„i., ^^^^ ^^^^ 

ronowcd , .n. . 3nd .Hen , ..ain. os ... dcscr.Hcd in conncc.on w,H hc.inor.c. discussed Fi,. 3. T.us 

=.cH Ccui. is co.p,c.H. ,,nch.nous. THi. .c.Hod. howev„. H.s in.insic ,i.i,3,ions wi.H b.nd.id.H u.i.i.a.on. 

s.n« if 3 circi. Hns no.hin, .o send i.s aHoca-ed b.ndwid.h is no. used on ..e .inc. addresses .his bandwid.h 

i«ue b, a„owin, .He ci.ui.s .o be asyncH.nous. THou.H b.ndwid.h is s,i„ divided o.ong Hxed .eng.h d.a i.e.s. 

any circuit can Uansmil a. any poin. in time. 

The ITU T (InicmaiionnI Telecommunications Union - Telrrnrr,,^. • • 

aiions union Telecommunications, formally the CCITT). is an 

organization chanercd by .he Unilcd Nniions to nrnvM^ i^t^, • • 

rsn.ions to provide telecommunications standards defined four classes of 

service: I) Consiani Bit Raie for Circuit Emulation ic eon5i3ni r,„. ■ ^ 

. . . cons.ant-raie voice and video; 2) Variable Bit Rate for 

.nd video npp,icn.ioo. 3) Daio for Conneetion-Or.entcd Trnffic: and 4) Dat. for Conn. 



Oriented Trafnc. ^,ese services, in turn. a. supported by cer^ain -.lasses - of AT^nraffic. a™ r. 

Layers (AAL). these ATM adaptation layers are d-fined in ITI I T D 

yers d.nncd ,n ITU-T Reeommendalion 1.363. TTiere are 3 defined 

AAL . „„«.a .„,o ™= ,„ ,™ „„ ,^ _^ 

btlonj m „„c ,^ „pp„„d lo cell, ,ho, bctong ,o .noikcr lr,,c,. 

Ihc .d^n.-vlon !,,„ i, *K,„,i„,<, d„,i„E ci,c„i, ,„„p; i.,. , ^^^^^^^ „ 

network. At this time, the hn^l cninpuicr inrnrTn<: tHe net wort nfil... u. - -. -.i 

tne network of the layer it will use for a specific viniial circuit 

AAL, H.,. *r... ,„ ^ r„ „ „ ^^^^ ^^^^ 
r™ u„ d™,„„ „„„„ ,^ ^^^^^^^^^ ^ ^^^^ ^^^^^ 
.cr,„. . ^ ^^^^^^ ^ 

'-'-^-'"'-'■''^'-''^-'^'-^.^^^^^^^^^^^ 

nr through ATM neiwr,rks. If connected dircctlv. then packet, are =rh;,„ • u -r 

packet., arc arbitrary $,7.c: but if connected by ATM switches 
then all packet., ex.tinp the rouicr .nre chopped into n.xed si« cells of 53 by.es. 
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Network architectures based on the Internet Protocol (IP) technology are designed as a "best effort" 
service. This means that if bandwidth is available, the data gets through. If. on the other hand, bandwidth is not 
available then the data is dropped. This works well with most computer data applications such as file transfers or 
remote terminal access. This does not work well with applications that can not retransmit, or where retransmission is 
of no value, such as with video and voice. Celling a video frame out of order makes no sense, whereas Tile transfer 
applications can tolerate such anomalies. Since the packet size is arbitrary at any point in time making specific delay 
variation commitments between any two frames is almost impossible, as there is no way of predicting what type and 
size of iraffic is ahead of any other type of traffic. The buffers that must handle the data, moreover, must be able to 
receive the maximum data size, meaning that that buffering scheme must be optimized to handle larger data packets 
while al the same time not wasting loo much memory on smaller packets. 

ATM is designed to provide several service categories for different applications. These include Constant Bit 
Rate (CDR). Available Bit Rale (ABR). Unspecified Bit Rate (UBR) and two versions of Variable Bit Rate (VBR). 
real-time and non-real-time. These service calegories are defined in terms of Traffic Parameters and QoS Parameters. 
Traffic Parameters include Peak Cell Rale (constant bandwidth). Sustainable Cell Rate (SCR). Maximum Burst Size 
(MBS). Minimum Cell Rale (MCR) and Cell Delay Variance Tolerance. QoS parameters include Cell Delay 
Variation (CDV). Cell Loss Ratio (CLR) and ma;?imum Cell Transfer Delay (maxCTD). As an example. Constant 
Bit R.nic CBR (e.g. the service used for voice and video applications) is defined as a service category that allows the 
user ai cM .<;clup lime to specify the PCR (peak cell rate, essentially the bandwidth), the CDV. maxCTD and CLR. 
The network must then ensure that the values requested by the user and accepted by the network .are met; if they arc 
met. the network is said to be supporting CBR. 

The various classes of service direct the network to provide belter service for some traffic as opposed to 
other types of traffic. In ATM. with fixed length cells, switches manage bandwidth utilization on a line effectively by 
controlling the amount of data each traffic flow is allowed to put on a line at any moment in time. They generally 
have simpler buffer techniques arising from the fact that there is but one size of data unit. Another advantage is 
predictable network delays, especially queuing latencies at each switch. Since all data units are ihcsame size, this 
helps to ensure that such traffic QoS parameters as CDV are easily measurable in the network. In non-ATM 
networks (i.e. frame-based networks), frames can range anywhere from, say, 40 bytes to thousands of bytes, 
rendering ii difficult to ensure a consistent CDV (or PDV. Packet Delay Variation) since it is impossible to predict 

-3- 
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(he delays in tht nciwork. lacking consistent transfer limes of individual packets. 

By carving data into srriallcr units. ATM can increase the nhihty of the network to decrease the latency of 
■r.msmitting data from one host tn nnothcr. Such also allows for cnsicr queue and buffer management at each hop 
through the network. A dis.idv .-.nt.-.pc. however, is .hat a header is added to each cell making the effective bandwidth 
of the network less than if the network had a larger transmission unit. For example, if 1 .000 bytes are to be 
iransferrcd from one host to .mother, then a frame-based solution would append a header (approximately 4 bytes) 
nnd transmit the entire fr..mc in less than n second. In ATM. the 1 .000 bytes is chopped into 48 bytes with a 5 bytes 
header: i.e. I.0OO/4S = 20.833 (or 2 1 cells). Each cell is then given a 5 byte header increasing the bytes to be 
transmitted by 5 • 2 I = 105 e.tra bytes. Thus ATM effectively decreases the available bandwidth to the actual data 
hy approximately 100 hytcs (or .nbout 1 07.): the decreasing of end-io-end latency also decreases the available 
bandwidth for d.nia lr.Tn<;missinn. 

Tor some applications, .such .is video and voice, latency is more important than bandwidth while for other 
applicaiinnv such .is f.le tr.-msfcrs. better bandwidth utilisation incre.iscs pcrform.nnce rather than decreased hop-by- 
hop latency. 

Recently, the detnnnd.. on' more bandwidth and QoS have grown many fold due to new applications for 
multimedia services, including the hcfore described video and voice. This is forcing the growth of ATM networks in 
the core of trnditional pnckc.-h.nscd networks. ATM. because of its f,.ed packet size, brings reduced processing time 
in networks and hence f.Tstcr fnrw.-,rding (i.e. lower l.itency). It also brings with it the ability to take advantage of 
traffic classrficniKm. Smcc the cells, ns earlier pointed out. arc of fixed she. traffic patterns can be controlled through 
QoS assignments: i.e. networks can carry traditional packets (in cell formal) and constant bandwidth stream data 
(e.g. voice/video based data). 

As will subsequently he demonstrated, most conventional networking systems inherently are designed- for 
either forwarding frames or cells hut not both. In accordance with the present invention, on the other hand, through 
use of novel search alporith.nv QoS management and manngemenl of p.nckct/cell architecture, both cells and frames 
can be transmitted in the .same device and with significant advantage over the prior techniques, as later more fully 
explained 
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Objects pr Invention 

An object of ihe present invention, accordingly, is to provide a novel system architecture and method, 
useful with any technique for processing data packets and/or cells simultaneously with data packets, and without 
impacting the performance aspects of cell forwarding characteristics. 

A further object is to provide such a novel architecture in which the architected switch can serve as a packet 
switch in one application and as a cell switch in another application, using the same hardware and software. 

Still a further object is to provide such a system wherein improved results are achieved in managing QoS 
characteristics for both cells and data packets simultaneously based on a common cell/data packets algorithm. 

An additional object is to provide a common parsing algorithm for forwarding cells and data packets using 
common and similar techniques. 

Other and further objects will be explained hereinafter, and are more particularly delineated in the appetided 

Summary 

In summary, from one of its important viewpoints, the invention encompasses in a data networking system 
wherein data is received as either ATM cells or arbitrarily-sized multi-protocol frames from a plurality of I/O 
modules any of which can be cell or frame interfaces, a method of processing both ATM cells or such frames in a 
native mode, i.e. not transforming frames to cells, using common algorithms for forwarding based on control 
information contained in the cell or frame and in such a manner as to preserve QoS characteristics necessary for 
concci operation of cell forwarding; processing the packet/cell control information in a forwarding engine with 
common algorithms not dependent on context-sensitive information contained in the cell or packet, and passing 
results including QoS information to an egress queue manager; passing the cell/ packet to the egress 1/0 transmit 
facility in such a manner as to provide a minimal cell delay variation (CDV) so as not to impact correct cell 
forwarding characteristics; and controlling the Uansmit facility so as to provide a common bandwidth management 
algorithm for both cell and packets and all without impacting the correct operation of either cells or packets. 
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Drawings 

The invention will now be described in connection with the accompanying drawings in which the before- 
mentioned Fig. I is a diagram illustrating an ATM (Asynchronous Transfer Mode) cell format: 

Fig.2 is a similar diagram of an Internet Protocol (IP) frame format for 32 bit words; 

Fig.3 is a flowchart comp.iring Time-Division Multiplexing (TDM). ATM and Packet Data frame 
forwarding: 

Fig. 4 is a block diagram of the switch of the invention with the cell and packet interfaces; 

Fig. 5 is a block dingram of a traditional prior art bus-based switching architecture, and Fig. 6. its memory- 
based switch data flow diagram: 

Fig. 7 is a block diagram of a traditional prior art cross-bar type switching architecture, and Fig. 8, its cross- 
bar data flow diagram: 

Fig. 9-10 arc interface diagrams illustrating, respectively, a cell switch with a native interface card, a packet 
interface on cell switch, and an A.-\L5 packet interface on cell switch, all with a cross-bar or memory switch; 

Figs. 12 and 13 are similar diagrams of a packet switch with native packet interface cards and with AAL5 
interface, respectively, for NxN' memory connection buses; 

Fig. 1 4 is a block diagram of the switch architecture of the present invention, using the word "NeoN" in 
connection with the packet and cell data switch as a trade name of NeoNET LLC. the assignee of the prescnt 
application: 

Fig. 15 and 16 are diagrams respectively of extended parsing function (lows for forwarding decisions and 
an overview of such functions and Fig. 17 is a diagram of the forwarding elements; 

Fig. 18 is a first stage parse graph tree lookup block diagram, and Fig. 19 is a second stage forwarding ubie 
lookup (FLT) diagram; 

Figs. 20 and 21 arc respective diagrams of parse graph memory on power up and of a simple illustrative IP 
multicast packet; 

Fig. 22 presents an initialized lookup table, with all entries pointing to unknown route/cell forwarding 
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informaiion. and Fip. 23 illuMraic-; ihc lookup lablc aficr addine an illu^in.lv^ id 

aaoing an iMustraiivc IP address (209.6.34.224/32): and 

Tig. 24 is n queuing di.ngram for scheduling sy.Mem operation. 
Further nackgrnund To Pr ererred Emhndiment.'; nf tnvpn..n„ 

Before proceed.ng ,„ i„us,ra,= .h= preferred architecture of ,he invention, i, is believed necessary .o review 
.he limitations of the prior and of cur.cn, network systems, which ,hc prcscn, Invcn.ior, admirably overcomes. 

Current networking solutions arc designed cither for switching data packets or cells. As before su.ed. all 
types of data networking switches r.u..t receive data on an ingress port. r.ake a forwarding decision, transfer data 
from the ingress por, to the egress port and transmit that data on the .appropriate egress port physical Interface. 
Beyond the basic da,.. forw.-.rd,ng aspects, there are different requirements for cell switching versus frame 
rorw.r.,ng As hefore stated, a,, current technology divides switching elements into three types: bridges, routers and 
switches, and in particul... ATM switches. T^e distinction between bridges and routers is blurred in that both ' 
forward d..:agrnms and .ypicallv mas, routers also do bridging funcions as welL-thus the discussion focuses on 
datagram switches (i.e. routers) .nnd ATM switches. 



It is in order firs, to 
based on current solutions, and then to present the reasons why current 



le the basic architectural requirements for these two types of switching devices 
do not provide mechanisms to allow 
simultaneous transfer nf cells and frames without severely impacting the correct operations of either ATT^I switching 
or frame forwarding. n,e novel solution based on the present invention will then be clear. 

Routers typically h,.ve a wide variety of physical interfaces: LAN interfaces, such as E.hern'et. Token ring 
and FDD., and wide-area .nterf..ees. such Frame Relay. X.25. T. and ATM. A router has methods for receiving 
frames from these various interfaces, and each interface has different frame characteristics. For example. .„ Ethernet 
frame .ay be anywhere from 64 bytes .0 1500 bytes, and an FDDI fratne can be anywhere fron, 64 byt« ,o 
4500(incMing header and trailer) bytes. TT,c router's I/O module strips the header that is associated with .he 
physical interface and presents the resulting frame, such as an IP datagram, to the forwarding engine. Tl,e forwarding 
engine looks a, the IP destination address. Fig. 2. and makes an appropriate forwarding decision. "Hte r«uU of a 
forwarding decision is to send datagram to the egress port as determined by the forwarding tables. Tte egress port 
.hen att.,ches the appropriate network-dependent header and transmits the frame out the physical interface. Since 
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different interfaces may have different frame size requirements, a router may be required to "fragment" a frame, i.e. 
"chop" the datagram into useable size. For example, a 2000 byte FDDI frame must be fragmented into frames of 
1500 bytes or less before being sent out on a Ethernet interface. 

Current router technology offers "best effort" service. This means that there are no guarantees that 
datagrams will not be dropped in a router-based network. Funhermorc, because routers transfer datagrams of varying 
sizes, there are no per datagram delay variation or latency guarantees. Typically a router is characterized by its 
ability to transfer datagrams of a certain size. Thus, the capacity of a router may be characterized by its ability to 
transfer 64 byte frames in one second or the latency to transfer a 1500 byte frame from an ingress port to an egress 
port. This latency is characterized by last bit in, first bit out. 

An ATM switch, by comparison, has only one type of interface, i.e. ATM. An ATM switch makes 
forw.nrding decision by looking at a forwarding table based on VPLA/CI numbers. Fig. I . The forwarding table is 
typically indexed by physical port number, i.e. an incoming cell with a VPI/VCI on ingress port N gets mapped to an 
egress port M with a new VPIA/Cl pair. The table is managed by software elsewhere in the system. All cells, no 
matter what the ATM Adaptation Layer (AALx), have the same structure, so that if ATM switches can forward one 
AAL type, they cnn forwnrd any type. 

In order to switch ATM cells, several fundamental criteria must be met. The switch must be able lo make 
forwarding decisions based on control information provided in the ATM header, specifically VPIA'CI. The switch 
must provide appropriate QoS functions. The switch must provide for specific service types, in particular Constant 
Bit Rale (CBR) traffic and Variable Bit Rate (VBR). CBR (voice or video) traffic is characterized by low latency 
and more importantly low or guaranteed Cell Delay Variation (CDV) and guaranteed bandwidth. 

The three main requirements of implementing CBR type connections over a traditional packet switch are 
low CDV, small Delay and guaranteed bandwidth. Voice, for example, consumes a fixed amount of bandwidth, 
based on the fundamental Nyquist's sampling Theorem. CDV is also part of a CBR contract, and plays a role into the 
overall Delay. CDV is the total worst case variance in expected arrival time and actual arrival time of a packet/cell. 
In so far as an application is concerned, it wants lo sec data arrive equidistant in time. If. however, the network 
cannot guarantee this equidistant requirement, some hardware has to buffer data - equal or more than the worst case 
CDV amount introduced by the network. The higher the CDV, the higher is the buffer requirement and hence the 
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higher Delay: nnd. ^ iIluMr.-„cd cnrl.cr. Dclny is not good for CDR type circuits. 

P..c.e..h3scd nctwor.. .r.d„lon.,v ,..uc d.3 . the egress boscd on priority of tramc. Rcgordlcss of how 
d..n is ,.cucd. trofnc . i.h low dclny vnrin.ion rccuircmcr,.s wi„ gc queued behind one or n,orc packets. Each of 
.hem could be ...„num pnc.et ,i.c. nnd this inherently contributes the mos, to deby v.ri.ion oa a pac.et-based 



letwork. 



-n^ere many rT,e,hodologies used to manage bandwidth and priorities. From a Network Management 
point of view, a ncwck m.,n.-,gcr usually l.kes to carve out the total egress bandwidth into priorities, "n.ere arc 
several reasons for carv.ng th.s b..ndw,d.h: e.g. i, ensures the manager that control traffie (Higher Priority and Low 
Bandwid.h) olv...ys has room on the wire even during very high line bandwidth utilization, or perhaps a CBR 
(Consi.mi Oil Rale) traffic will be guaranteed on the wire. etc. 

-n,cre are numerous methods to address bandwidth per trafHc priority. Broad classes of these mechanisms 
are Round Rob.n Queuing. Weighted Pair Queuing and Priority Queuing. Each methodology wil, be explained for 
.he sake of discuss.on and comple.ene.., of thi. document. .„ all cases of queuing. trafHc is put mto queues based on 
rr.or.t,es. usually by a hardware engine .ha. looks a. a cell/packe, Header or control information associated with 
cell/pncke. as .he cel.packet arrive., from the backplane. is how data is e.racted/de-queued from these queues tha. 
differentiates one queuing mechanism from another. 



Simple Round Robin Queuing 

Ihis queuing mechanism emp.ies all queues in a round robin fashion. TMs means that trafHc is divided into 
.ueue. and each queue gets the same H.ed bandwidth. While a clear advantage is simplicity of implementation, a 
n,.or d,.dvan.age of this queuing technique is that this mechanism comple.e.y loses the concept of pHorit,. Priority 
n^ust then be managed by buffer allocation mechanisms. "Hte only clear advantage is simplicity of implemenu.ion. 
Weighted Round Robin 

TXis queuing mechanism is an enhancement of -Simple Round Robin Queuing •. .here a weigh, is placed 
on each queue by the network manager during initialization time. ,n this mechanism, each priority queue is serviced 
hasedon the weigh,. ,f one queue is allocated 10% of the bandwidth, i, wil, be serviced .07c of .he .ime Another 

queue m..yhave 50. Of the ..l,oc..tedb..ndwid.h. and willbe serviced 50.of .he .ime.T.e.3iord^^^^^^^^^ 

IS unused handwidih on the wire when there i^nn.r.rr • 

IS no trafHc m a queue of the allocated b.nndwidth. This results in wasted 



bandwidlh There is moreover ■ • 

■no association of packet size in the de-queuing algorithm, which is crucial for 
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r3c.c....ed .vi.c.cv a.;., .ci,h. .0 an pocKc. si.cs ,h.o.. off .h. bandwid.h .Uoca.icn schc.c. 

Priority Qucuinc 

m .h,s cuco.np -ncchnni.n.. ou.pu, quc.cs nrc serviced purely based on priority, "n-e HigheM Pncri.y 
Q.e.e ,e,s ser. iced Hrsc. ar,d .Ke Lowes. Priori.y Queue ,e,s serviced ,3... ,r, .his mechanism. Higher PHori.y 
Trarnc aKvay. preer^p.s .He Lo.er Priori.y Queue. The drawback of .his .ypc of mechonism is .ha. ,hc Lower 
Priori,. N,ech..ism m,v re.,, in .rr, band.id.h. I.e advan.a.e of .his rr,echanism. Besides bein, simple, is .h« 
h.nd.,d„. ,s n„, ...... so ,on, a, .here . da.a .0 send. i. wi„ ,c sen,. ..ere is. however, no assoeia.ion ofp^.e. 

de.q.culne algorl.h,,,. w„ieh is crucial for packe.-based swi.che.v Giving equal weigh. ,0 all packe, sizes 



off Ihc bandwidth : 



IS before noicd. 



rrcr. ,he above cnmples. .here is a need .0 s.n.e a balance be.ween Priori.y Queuing and Weigh.ed 
Round Rnbin Queuing, ainng wi.H packe. si.e. ..is calls for a solution provided by .He prescn. invention where . 
hi.H pr.nrity trafHc is scrv.ccd bcft.c lower priority .rafOc. but e..ch queue is serviced 3. leas, within i.s bandwid.h 
.Mocon. in addnion .0 .He above requirement, .he output buffer should be filled wi.h data from a queue even when 
.he b..ndw„.H of that queue ,s exhausted, inc.ud.ng with other bandwidth eligible queue data. T^is .echniquc 
enforces bandwidth per trafOc queue requirement and also does not .as.e bandw.d.h on .he wi. and is embodied in 
the invention 

Architectural I.s.suc.<; in Switch Design 

Current s w„ch,ng solutions employ . wo dis.inc, solutions: I ) memory and 2) cross-bar. Tl^ese soluiioris «c 
Hlustrated in P.g. 5 and 6 .showing a traditional bus-based and memory based archi.ec.ure. and in Pig. 7. showing . 
tradi.ional ctoss-b.ir swi.chlng nrchitcciure. 

in .he .radi.ionnl men.orybased so.u.ions rcprcscn.ed bv Fig. 5. da.a .us. firs, be placed inside of main 

memory from the l/Q c.nrd. Thh d.it.T .r.insfer .akes several cycles hi.. 

several cycles a.s bi.s are moved from the I/O module lo ihe main 

r^emory. Since several dif feren. ,/0 modules mus, transfer da.a ,0 common memory, con.en.ion for this resource 
occurs. Main memory provides both a buffering mechanism and a transfer mechanism for data from 0. physic.. 

Por..oanotherphysicalpor,...cra.eof transfer is .henhigh,vdependen,on.hespced of thecgresspor..„d the 
shilitv ol the sv^lcm 10 move ri-in in r 

-n and out of matn memory and .he number of interfaces .ha. must access rrain 
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memory. 

As more fully shown in Fig. 6. ihc CPU interfaces through a common bus. with memory access, with a 
plurality of daia-receiving and transmitting I/O pons #1. «2. etc.. with the various dotted and dashed lines showing 
the interfacing paths and the shared memory, as is well known. As pointed out previously, the various accesses of 
Ihc sha/ed memory re.^ult in substantial contention, increasing the latency and unpredictability, which is already 
substantial in this kind of architecture because the processing of the control information cannot begin until ihe entire 
packet/cell is received. 

Furthermore, a.'; the .icccsscs to the shared memory arc increased, so does (he contention: and as the 
contention is increased, this results in increasing the latency of Ihe system. In the traditional memory-based switch 
data flow diagram of Fig. 6. thus, where the access time per read or write to Ihe memory is equal to M. and (he 
number of bits for a memory access is W. the following functions occur: 

There is the write of data from the receive port #1 to shared memory. The time to transfer a packet or eel! is 
equal lo ((D •8)AV)-M. where B is equal lo Ihc number of bytes for Ihc packet or cell. M is the access time per read 
or write to the memory and W i. th- number of bits for a memory .-.ccess . As the packet gets larger so does the time 
to write it to memory. 

Thi.s means th.il if n packet is destined to an ATM interface as in Fig. 5. followed by a cell, the cell is 
delayed by tl.c amount of tr.-msfer lime from main memory, and in the wor.st case this could be N packets (where N is 
the number of packet. non-A TM interfaces) including the contention among other reads and writes on Ihe bus. If. for 
example. B=^00O bytes and M is 80 nanoseconds (for a 64 bit-wide bus for DRAM access), then ((4000 • 8)/64) • 
80 = 40.000 nanoseconds for a packet transfer queued before a cell can be sent, and OC 48 is 170 nanoseconds per 
64 byte cells. This is only if there is no contention on the bus whatsoever. In the worst case, if a switch has 16 pons 
and all the ports are contending simultaneously, then to transfer the same packet would require 640.000 nanoseconds 
just to get inio the memory, and the same amount lo get out- a total lime of about 1 .3 milliseconds. This occurs if 
between c.ich write into memory, another port has lo write lo memory as well. So for n= 1 6 ports, n-l. or 15 ports 
have to gain access to men.ory. Thi.s means ih.nt 15 ports « 80 nanoseconds = 1200 nanoseconds are-used by the 
system before the ncxl transfer into memory of the original porl can occur. Since there are '4000 byles • 8 
hits/byicV64 bits = .S()0 ncccs..c.s. e.-,ch access is separated by 1200 nanoseconds, and the full transfer lakes 500 • 
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1200 = 600.000 nanosecond. So ,h= ,o.al is sys.e^n ,lmc plus ncuol .ronsfcr ,in,= which is 600.000 nanoseconds . 
^0.000 n.-,nosccond. = r..0.000 nnnoscccnds for ,hc .r..n..fcr in.o n^cmory. and ono.hcr WO.OOO nanoseconds cu, of 
memory. This cnlculniion. moreover, docs no. include any CPU con.cn.ion issues or delay because of egress pon 
bu.'iy. v^hich would m.ikc ihis c.ilculaiion e\cn larger. 

There arc .vmil.ir disndvantngcs in iradiiional cross-bar based .solutions as shown in Fig. 7. before 
referenced, where .here i. no main memory-, and buffering of da.a occur.s bo.h at the ingress pon and egress pon. In 
(he memnry-hased design of Figs 5 and 6. buffer memory is shared across all pons, making for very efficient 
u.iliza.ion of memory on the .u i.ch In the cross-bar approach of Fig. 7. each port must provide a large amount of 
memory, so .ha. .he over.->ll memory of the .system is large as there is no common sharing of buffers. TTie cross-bar 
switch IS only a conduit for the ir.i.nsfcr of daia from one physical per. on .he system to another physical pon on the 
system. If two pons a,c sl.nuli.incously to tr.msfcr d.i.n lo one ou.put pon. one of the two input ports must buffer the 
data thereby increasing ihc latency and unprcd.ctabilily as the data from the firs, input port is transfened to the 



oulpul pon. The ndvaniagc of a cross-bar s 



h over a memory-based switch, however, is .he high rate of data 



transfer from on; poini 10 anoiher wiihoul Ihc inherent limilalion of mnin memnrvrnnr^n.;,, .t, l j 

Miiijiiuii ui main memory contention on the memory-based 

switch. 

In the iradi.ion.M cross-bar switching architecture system of Fig. 7. the CPU interfaces through a common 
bus. Will, memory access, in an inicrf.icc wiih the various dotted nnd dashed lines of Fig. 8 showing the interfacing 
paths and the shared memory, as is well known. Tlie CPU makes a forwarding decision based on information in the 
data. Tlic da.a must .hen he .,ans,n,i.ed across the cross-bar swi.ch fabric to the egress por.. But if other Iraffic is 
being forw.-.rded to th.i. egress interface, then the da.a must be buffered in the ingress interface for so long as the 
amount of tim.c u ..ikcs In transfer .he entire cell/packet to the egress memory. There is: 

A. Write of data from the receive port « I to local memory. The lime to transfer a packet or cell is equal 
to ((B*R)AV)«M. where D i.<: equal to the number ofbytcs for the packet or cell. M is the access lime 
per read or write to the memory and W is the number of bits for a memory access . As the packet gets 
larger so docs the lime lo write it lo memory. 

n. Write of da.a from the receive pon « I to local memory of egress pon #2. The lime to transfer a 

packet or cell is equal to ((B-R)AV)-M + T. where D is equal to the number of bytes for the packet or 
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cell. M i. ,hc .ccc.s lime per read or write to .h= rricmory. W is ,hc riurr^bcr of bi>s for a memory 
nccc.s end T . the transfer time of the cross-bnr .witch.. As the packet gets larger, so docs the time to 
transfer ii ncrn^-; the cross bar switch and write it to locnl memory. 
Por a packet .rnn.fcr followed hy a cell transfer to an egress port, the calculatior, is the same as for the 
memory.bnsed solution of Fig.v .S and 6, The packet must be transferred to local memory at the same speeds as for 
the memory-based solution l^c .dvan.nge that there is no contention for central memory, does not alleviate the 
problem that a packet transfer in front of a cell transfer can cause delays that prevent the proper functioning of very 
fast interface speeds. 

■n,e goal is to create a switching device running at high speeds (i.e. SONET defined rates) that provides the 
required QoS. 1,.. device ..hculd he scalable in terms of speed nnd ports, and the device should allow for equal-.ime 
transfer of cells and frnmcs from an ingress port to an egress. 

While current design.. h..vc M.-uted to come up with very high speed routers, they have not. however,been 

able to provide all the ATM service rcQulrem-nis ihn. ulll n,-,;M,.,;„- i • j 

I.C rcqjirem.nis. thus still maintaining n polarized set of networking devices, i.e. 

routers and A^ switches An opiiinnl solution is one that achieves very high speeds and that provides the required 

QoS supper, and has in,c,f.v-c.. th.., merge ATM and Packet-b.^sed technologies on the same interface. Fig. 3. This 

will allow the current invcMmcn, in either networking technology to be preserved, yet satisfy bandwidth andQoS 

demands. 

■n.e issues in nierginc interfaces on a data switch port that accepts ATM cells and treats certain ATTvl cells 
as packets and others as ATM Hows. ..ccepts only packets on other interfaces and only cells on yet another set of 
interfaces, is shown in later-di.^cusscd Fig. 4. T^cse issues are three fold: a) Forwarding decision at the ingress 
interface for packet and cells, h) su itching packet and cells through the switch fabric and. c) mar,aging egress 
bandwidth on packet and cells. The present invention, based on this technique of the previously cited co-pending 
applications, explains how ,o create a general data switch that merges the two technologies (i.e. ATM switching .nd 
packet switching) and solves the three issues listed above. 

Interface Issues Switch Designs 

■n.e purpose of this section is to compare and contrast ATM and Packet-based switch designs and various 
.nterfaces on either type of su itch design. Specifically it identiftes problems with both devices as they peruin to forwarding 
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r^ckc. or ccl!.: i c. is.ucs u iih AT>.1 sw„chc. forwarding packc,.. nnd issues wi.h P.ckc. s^vi.chcs forwarding cells. Fig. 
3. 

Typical Design of an ATM Swiich 

As prcvioaUy cpbincd. dcOn.d wi.hin .he AT^I s.nndnrd .here arc multiple ATM Ad.p.ation Layers ( AAL I- 
AAL5, . each one spcc.fying a d.ffcren. type of service from a wide spccrurr, of services: nan,ely. Cons.an. Bit Rn.c (CBR) 
10 Un.pecincd nit R.,e (UBR). Cons.am Bic Rn.c (AALl ) comraci guarnn.ees n,inimal cell loss wi,h low CDV. while 
Unspccined Bi. Rn.e con.rac, specif.es no .rafHe parnme.crs. and no quality of Service guarantees. For the purposes of .his 
invention it i^ convenient to limit the discussion to AALl (CBR) and AAL5 (Fragmented Packets). 

F,g 9 illustrates cell switching with native cell interface cards, showing different modules of a generic ATM 
Switch with nat.ve ATM interfaces. The cells arriving from the physical layer module (PHY) are processed by a module 
called Policing Function Module, which validates per VCI established contracts (services ) for ii 



ing cells: e.g Peak 
parameters such as Cell Delay Variation (CDV) and Cell Loss 



Cell Rate. Sustained Cell Rate. Maximum Burst Rate. Other 
Rate (CLR) arc guar.-,ntces provided by the box based on the actual design of the cards and the switch. TT^e contracts are se, 
by the network manager or via ATM signaling mechanisms. Cell Data from the policing function then goes, in lh= 
example of Fie 9 .to a Cross Bar-type (F.g 7, or Memory-based Switch (Fig 5), Cells are then forwarded to the egress port 
which has sotne requirements of shaping trafHc to avoid congestion on the remote connection. To provide egress shaping, 
.he design will have to buffer d.v.a on the egress .side. Since ATT^I connections ore based on a point-to-point basis, .he 
Egress shapcr module also has .o translate the ATTvI Header. TT^is is because .he nex. hop has no relationship to .he ingress 
VCWPI. 

Native Packet Interface on ATM Switch 

As mentioned In ,hc 'Background' section, if an AT^^ switch is lo provide a method that facilitates .he routing of 
packets, .here have to be a, le.ns, two points be.ween two hosts where p..ckets and cells networks mee.. ■n,is means .hal 
current cell switching equipment h.ns to carry interfaces th.nt have native packet interfaces, unless .he swi.ch is si.ting deep 
in the core of the ATM network. 1, is now in order, therefore.' to examine .he design of such a packet interface .ha, connects 
10 the ATM switch. 

A typical P.-,cke. interface on nn ATT^t Switch is shown in Fig. 10. elaborating on pnckc, interface on the cell 
switch, -ntc physical interface would put incoming packets into a buffer and .hen they arc fed ,o the "Header Lookup .nd 
Forwarding Engine", "n^e p.^cket-bascd forwarding engine decides the egress port and associates a VCI number for cells of 
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.ha, pnckc. -n,c p=ckc< .hen gc, scgn^cn.cd into cells by ,hc Scg:ncn,n,ion Unit. From there or,. ,hc pockc, is treated jus. as 
in the native Cell Switching co.e. which involves going through a policing function and to the Switch Buffer before entering 
the switch. On the egres. sule. if the cells enter a cell interface, then the processing is jus, as explained above (in the native 
cell in.erf..ce on ATM switch). If the cells enter a packet interface, then the cells have to be reassembled into packets. TT,ese 
packets .nrc then put into various priority queues and then emptied as in the packet switch. 

Two types of p.ickel interfaces on the ATM Switch should be examined. 
AAL5 Interface on ATM Switch 

A Router connected to ATM Switch could segment packets before sending the packet to the Al^i Switch. In that 
case, packets would anive at the ATM Switch in AAL5 format, before described. If the ATM Switch were ,o act as a 
Router and an ATT., Switch, it would h.nve to reassemble the AAL5 Packet and perform a routing decision on it. Once the 
ATM Switch/Router n.akcs the forwa.d.ng decision on the AAL5 packet. ,t would then push it through the ATT.1 Switch 
after segmcnlinp it again. 

In AAL.S. perfect interface on an ATM Switch .s shown in F.g. ■ 1 . Incoming AAL5 cells are Hrst policed on a per 
VCI based to ensu,. th,.t the sender is honoring the contract. Once the policing function is done, an Assembler will 
assemble the cells of a VCI into p.nckc.s. These packets are then forwarded to the forwarding engine, which makes the 
forwarding decision on the a..sembled packet and some routing algorithm. T7,e packet then travels the ATM Switch as 
mentioned in the P.nckct Interface on ATM Switch section, above. 

Difficultic; in Proce.vsing Packets on Cell Switch 

Keeping the goal of .he present invention in mind. i.e. to achieve strict QoS parameters such as CDV and latency 
and packet lo.s. this section will ,iM „-.c dirHcuIties of attempting to design for packets through a traditional cell switch. 

According to Fig. U. once the incoming AAL5 segmented packets arc assembled and a for^varding decision is 
made, they are resegmented in the -Segmentation Unit". Across the Switch, the AAL5 cells are then reassembled into 
packets before they are shipped on the egress wire. TT,is segmentation and reassembly adds to .he delay and unpredictable 
and ttnmea.surablc PDV (Packet Dclny Variation) and cel. loss. As earlier mentioned, for packets ,o be provide QoS. i. 
would need to support contract that includes providing measurable PDV and delay. Delay is caused due to the fact .he cells 
have ,0 be rea.v.cmbled. Each reassembly would have to. In best case, buffer an entire packc, wonh ofda.a before wiling h 
complete and sending it ,o the QoS section. For , 8000 byte packet, for example, this could result in 64 usee delay in 
buffering on a IGig.nbi. switch. 



wo 99/35577 



PCT/IB98/01940 



Tnc PDV fo, . pnckc, .hrougl, a cell svvi.ch is even more of a concern than ,he oddi.ior^al delay. 7>.= asserribly 
process cnn be processing multiple p.ncket. c, >hc snrrie ,.me from various ingress ports nnd packets, nnd this couses an 
unpred:c.ab!e amount of PDV. essen,i..lly hnscd on .. itch contention and the number of retries of sending cells from 
ingress tn egress. 

Cell loss through .he switch c.uses p.-,ckets to get reassembled incorrectly nnd therefore adversely affects 
opplications that are real-tin.c content speciHe. Most file transfer protocols do recover from a dropped packet (due to 
dropped cells), but it c.iu.scs more traffic on the switch due to retransmissions. 

In summary, passing packets through .in ATM switch doe.s not provide packets with the same CDV and latency 
characteristics as cells. It simply provides a mechanism for passing a packet path through a cell switch. 

Design of Packet Switch 

A tradi,ion..l Packet Switch i,, shown i„ Fig. I 1 with nat.vc packet interface cards. Packets are forwarded to the 
Forwarding Eng.nc vi.n the physical interface. ■n,e Forwarding Engine m-.kes a routing decision based on some algorithm 
and the header of the packet. Once the egress por, is decided, the packet travels to the egress via the Packet Switch, which 
could be designed in one of many w.ys (e.g. N by N busses, large central memory pool. etc.). On egress, the packets end up 
on different traffic priority Queues, llitse Queues are responsible for prioritizing trafftc and bandwidth management. 

Cell Interface on Packet Switch 

The traditional p.icket swi.cli. shown in Fig. 1 3 with AAL5 interf.tces. provides a mechanism to allow cells to pass 
through the box so long as the cells are of AAL.S type. There is no practical way of creating a vinual cell switch ihrotigh a 
tradition.-il packet switch, nnd p.irt nf the present invention deals with the requirements of .such an architecture. 

After AAL5 cells arc policed for contract agreements, they are assembled into packets by an Assembly module, 
-nie packets thus created are then proces.scd exactly like native packet interfaces. On the egress side, if packets have logo 
out of the Switch as AAL5 cells, ihcy arc first segmented and then header ir.inslated. Finally they arc sh.nped and sent out. 
Dimcuhics in Processing Cells on Packet Switch: 

There are problems that a cell How faces as it traverses a traditional packet ..witch. It is extremely dimcuh for . 
traditional data switch, such .ns a router, to support the QoS gu.trantees required of ATM. To illustrate the point, refertnce 
is made to the diagram ..hown in beforc-descrihed Fie. 13. One of the biggest challenges for a packet swkch is to support 
AALl cells. The simple rc.xson is th.it the traditional Packet-based header Lookup and Forwarding engirtes do not 



wo 99/35577 



PCT/1B98/0194O 



simultaneous recognize cell., and packeis: therefore. AAL5 cells which can be conven.H " . 

convened inio packets are supponed. This is 

a severe restriction in ihe capahility of the switch. 

Among the feaiurc; of cclK rre the CDV anri th^ H,i,„ -I. • • 

CDV and the delay charac.er.s.tcs. Pushing cells through a traditional packet 

sw.tch adds rriore delay and an unpredicable CDV. The packet switch as is inherent in i, • .- 

' in Its name, implies that packets of 

various si.cs and number., are queued up on the switch. Pncketized cells would then have no chance of ma" , ' ' 

• ■avc no cnance of maintaining any type 

of reasonable QoS through the switch. 



Preferred Embodimp ntf.s) of the rnvpnfmn 

The present invention, cxemplarily illustrated in Figs 4 and 14 and until., ,ii .t. 

rits. ana i^. and unlike all these prior systems. 

optimizes the networking system for transmitting both cells and frames without internally converging one into the 
other. Furthermore it maininin< ih,- tir;^i r\„c 

the strict QoS parameters expected in ATM switches, such as strict CDV. latency 

and cell loss 17,is is achieved by having a common ingre.ss forwarding engine that is eon, , • . . 

"■life engine tnal is context independent, a switch 
fabric lh.1t transfers cells and frames with similar latency and a comm 

■cn.y, and a common egress QoS engine- packets Rowing 

.Hrough the architecture of the invention acquiring cell QoS characteristics while the cells still maintain their QoS 
characteristics. 

Tli: main components of the novel switch archii-cture of ihr in,., ,• 

arcnii.cture of the invention, sometimes referred to herein by the 

po„. -n,. p„, , „, ^^^^ ^ ^^^^ ^ ^^^^ ^^^^^^^ 

r.„Hc,„„. ™, ^ „„ ,„.,,„„^ . ^^^^^ ^^^^^^^^^^ ^ ^^^^^^ 

c=™™„ ,„ »„ p„ o„,„„ , ^^^^^^^^^ ^^^^ . 

common tn both cell and frame processing. 

,„ ^^^^^^^^^ ^^^^^^^^ ^^^^ 
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however, contains ..o phyMc.l interfaces AAL5/I and pnckc, interface a, the ir,gress end egress. "D^e difference 
between the two ,ypes of interface is the rr^odules listed as "Per VC Policing Function" and "Per VC Shaping". For 
cell interfaces (AALI-S). the system h.ts to honor contracts set by ,he network manager as per any AW switch and 
nho provide some sort of shaping on per VCI bases a. the egress. Besides those physical interface modules, the 
system is identical for a packet or cell interface. The system is designed with the concept ,ha. once the data 
traverses the physical interface module, there should be no distinction between a packet and cell. Fig. 14 lisu the 
core of the architecture which has three major blocks, namely. -Header Lookup and Forwarding Engine". "QoS". 
and. "Switch" fabric, that handle cells and packets indiscriminately. The discussion, as it relates to this invention. lies 
in the design of these three modules which will now be discussed in detail. 

Switch Fabric 

-Hie inventions presented m before-refcrenced co-pending U.S. patent applications Serial No. 581.467. and 

Serial Nn. 900.757. both of common assignee herewith omimizc the nriwnrWinacv., r ■■ ,. 

g -rc optimize ihcnciuorking system for minimal latency, andean 

indeed achieve ..ero l.ntency even .-.s dr,..-. rates and port densities are increased. They achieve this equally well, 
moreover, for either 53 byte cells or hy,e to 64K bytes packets through extracting the control information from 
the packet/cell as it is being written into memory, and providing the control information to a forwarding engine 
which win n.ake switching, routing and/or nitering decisions ..s the data is being written into memory. 

Native Cells through the Switch 

■n,e switch cells (AAL 1/5) of Pig. 14 are first policed at 2 a. per the contract the network manager has 
installed on a per VC! base, litis module could also assemble AAL5 cells into packets on selected VCI. Coming out 
of ,he policing function 2 are either cells or assembled packets. Beyond .hi.. Juncture of the data flow, there is r,o 
distinction between a packet or a cell until the data reaches the egress pon where data has ,o comply with the 
interface requirements, cells are queued up in the "NeoN Data Switch" 4 and the cell header is examined for 
destination in.erf.nce and QoS requirements. H^is information is p.nssed on to the egress interface QoS module 6 via a 
Control Data Switch, so-lahe.ed a, 8 .The QoS for a cell-type interface will simply ensure that cel. rate's beyond the 
Peak Cell Rate are clipped. "Dte cells are then forwarded to the "Per VCI Shaping" rr^odule 10. where .he cells .re 
forwarded ... ti.e physical in,e,f..ce after they are shaped as per the requirements of .he next hop switch. Since the 
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QoS .module 6 docs no. know f,om conuol data whchcr a pockc. or n cell is involved, i, si,.ply request. ,h= data 
fron, ,hc NcoN Swi.ch in.o .he -Buffer 12." Tne control da,a informs .he -Pcr VCI shopin,- block 10 ,o do either 
header .r.nsb.on if i. were . cell going in.o another VCI tunnel, and/or segmentation if .h= da.a was a packet going 
ou. on n cell inicrfncc and/or reform shaping as per .he remo.c end requirements. 

Native Packets through the NeoN Switch 

As pockets en.er the in,erf.-,ce card, .he packet header is examined by a Header Lookup and Forwarding 
Engine module 14 while .he da.a is sent .o .he NeoN da.a swi.eh 4. The Ingress For^varding Engine makes a 
rorwnrd,ng decision about .he QoS and .he des.ina.ion in.erfacc card based on the incoming packet header. The 
Forward.ng Engine 14 al.so gathers all information regarding the da.a packet, like NeoN Swi.eh address. Packet QoS. 
Egress Header Translation informa„nn. and sends it acro.ss to .he egress in.erface card. This information is carried as 
a con.rol packet .o the egress por. through the small non-blocking control data switch g .o the Egress QoS module 6. 
which will q.cue da.a ..s per .he con.rol packet and send i, ,o the module lis.ed PH V a. the egress. If the packet were 
to egress .o a cell in.erf.ace. ,he packe, will be segmented, then header .ransla.ed and shaped before i, leaves (he 

Advantages of the NeoN Switch Architecture of the Invention 

As seen above, cell and pacVe. Pnw through ihc box wi.hout any dis.inc.ion except at the physical 
interfaces, such .ha, if cell charac.cris.ics arc main.ained. then p.acke.s have the same characteristics as the cells. The 
packets n,ay .bus have measurable and low PDV (Packe. Delay Variation) and low latency, wi.h .he architecture 
supporting packe. .wi.ching wi.h cell charac.eriMics and yet interfacing ,o existing cell interfaces. 

While ,he traditional packe, swi.eh is un.able to send non-AALl cells as before explained. AAL5 cells .Iso 
suffer an unpredic.hle nmoun. of PDV and delay - ,his being obviated by .he NeoN Switch of .he invention. 
Packets through a traditional ATM Swi.ch also suffer ,he same long delays and unpredic.ablc CDV - again, no. .he 
case in ,he NcoN Switch of the invention. The modules that make this type of hybrid switching of .he invcn.ion 
possible include the Ingress Forwarding Engine, .he Egress QoS. and .he Swi.ch Fabric. 
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Ingress Forwarding Engine Description 

The purpose of >!,c Inprcv. Forwnrding Engine M. Fig. 14. i, ,„ parse ,hc input framc/ccll and. based on 
prcdcr.ncd criteria nod cnn.cnu of ,hc frnme/cell. make n forwnrdine .!eci.,ion. Thi., means .ha, ,h= inpu, cell/frnme is 
compnred agninsi i.em. .Mnred in memory. If a ma.ch is dc.ermincd. ihen ihe contents of the memory location 
provides command. fr,r action, on the cell/frame in question. The termination of the search, which is an iterative 
process, results in a forwarding dcciuon. A forwarding decision is n determination of how to process the 
aforementioned frame/cell. Such processing may include counting statistics, dropping the frame or cell, or sending 
the frame or ee!l to a sc. of spcciHcd egress ports. In Fig. IS. this process is shown a. a gross level. An input stream 
of four eharncers is shown h.cd.e. Tl^e characters have appropriate m.,.ching entries in memory, with a character 
input producing a pointer to the next character. IT.e f.nal character b produces a pointer to a forwarding entry. A 
different stream of characters than thn, illustrated would have a different collection of entries in memory producing 
different results. 

•n^e proposed Ingress Forwardrng Engine 14 i., deftned to be a Parsing Micro-Engine. 77,= Parsing Micro- 
Engine is divided into two parts - .-..^active part and a passive part. Tl.e active part is referred to as the parser, being 
log.e that follows instructions written into the passive memory component which is composed of two major storage 
sections: DParse Graph Tree (PCT). Fig. 18. and 2) Forwarding Lookup Table (FLT). Fig. 19. and a minor storage 
action for statistics collection 1 l.c Parse Graph Tree is storage area that contains all ,he packet header parsing 
information, the results of uhieh is an offset in the Forwarding Lookup. The FLT contains information about the 
destination pon. multicast information, egress header manipulation. The design is very nexible. e.g. in a datagram, it 
can traverse beyond the DA and SA fields in the packet header and search into the Protocol field and TCP Pon 
number, etc. The proposed PGT is memory that is divided into the 2" blocks with each block having 2" elements 
(where m < n ). Each element can be one of three types - branch element, leaf element, or skip clemen. and within 
each block, there can be any combination of element types. 

While ranicularly useful for the purpose of the present invention, the Parsing Micro-Engine is generic from 
the standpoint that i, examines an arbitrary collection of bits and makes decisions based on that comparison. This 
can be applied, for example, to any tcsl-searching functions. Scorching for certain arbitrary words. In such 
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.rP<i".ions. an ill.suo.ion. words such ns -bomb" or "dc.or^a.c" in . ,c,.er or email may be searched and if, 

ma,ch i, dc.cccd. ,hc sc^ch engine mny then execute predetermined functions such as signahng an alarm. In fac, 

the same memory con even be used to senrch fnr words in different languages. 

In the context of the invention. F.g . . illustrate., having two entry points. One entry point is used ,o search 

for text in one language, while the .second entry point is used to search for .ex. in another language. TT,us .he s,r.= 
mechanisms and the same hardware are used for .wo types of searches. 

-n^ere are two components to ,hc datagram header search, software component and the hardware 
component. The software component creates the elements in the Parse Graph for every new route i. finds on an 
interface. T^c software has to create a uni.ue graph starring from a Branch Element and ending on a Leaf Elemen.. 
b.er denned, for each additional new rou.c. The hardware walks the graph from branch to Leaf Element, clueless 
nhoul the IP hc.idcr. 

In fact there can be n.nny entry points in the memory region as illustrated in Fig. 21. Tl,e initial memory 
can be divided into multiple regions, each region of memory being a separate series of instructions used for differen. 



applications. In the ca.e of Pig. 22. one of.he regions is used for IP forwarding while .he other region is used for 
a™ forwarding At system s.... the memory is initialized to point to "unknown route", meaning that no forwarding 
information is available When a new entry is inserted, the structure of the Lookup Table changes, as illustrated in 
Fi. 23. Tl,e illustrative ,P address 209.6.3..224 is shown insened. Since .his is a by.e-orien.ed lookup engine, the 
r.rs. block has a pointer in.ser.ed in the 209 location. The pointer points .0 a block .hat has n new pointer value in .he 
6 location, and so on until el, of .he 209.6.34.224 address is inserted. All other values still point .o unknown route. 
Insening .he .nddress in the IP portion of memory hns no impact in .he ATM portion of memory. 
AS mentioned earlier, there are 2" blocks each with 2" elements in .he parse graph .rec. Tltc suucture of each clement 
IS as shown in Fig. 17. with each element having .he following fields. 

1. Instruction Field: In the current design there are three instructions resulting in two bit ins.ruc.ion Held. The 
instruction description is as follows. 

. Branch Elemen. ,00). In so far as .he Micro Engine is concerned, .he branch clement csscmi^ll^ points 
.he Forw..rding Engine to the next block address. Also, within .he branch clement, .he user may set 
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n„d, i„ ,„„™„„, p.™.„i„, „„„ . , , ^^^^^ ^^^^^ ^^^^^^^^ 

„, „.„ F„..,„;„, ,„r„,„„,„„ ^„ ^^^^^^ ^^^^^^^ 

»n .r ...e,. „a , ^, ^^^^^ ^^^^ ^^^^ 

p„, .„H. ,0,..,,, f„, ^^^^ 

cprcs-; port informniion ni the end of parsing (he VPI field. 

Lc3r E,c.en. ,0.,. -n.i. ele.cn, ins.rue.s ,Hc end of p...i„, .h= mie.o cn,i„e. T^c forw^ding 
inrorn,.ion .cco..,.ed d.ing .He se.cH . .Hen fo.w.ded .o ,He n„. ,o,ic3, Moc. in .He design 

Packet header depends on .he number of block addresses .he micro rno- u 

.he micro engine has lo look up. Not every 

„. „„„„ „ ^^^^ ^^^^^^^ ^^^^ 

d,,»p,.„„ n„d ,hc ,c„ch -n,, .lip ,i„ i, a„„i^^j 

dn.agramhcndernclds.o allow for r.McrsearchinE In an IP h. . r 

-ching. In an IP header, for example, if .he user wanted to forward 

packet ba^cd on DA but count s!-iisiics bn.rH nr, x c ,-r 

based on ToS (Type of Service) Held. i. would parse .He en.irc DA and 
Ihen siep lo .he ToS Field. Tnis makes for a fas.cr Forwa-din, P • ^ 

For.ard.ng Engine. The si« of .his field should be ealcula.ed 

10 allow for the largest 5l;ip thai the user would ever need for it, H-,, ■ . 

need for it.vd.-Ma switching box. which could be based on 

the protncol. etc. 

.. ..=.™=~. r.,.-.,.„, ,„,„ „„,.^ ,„,„^„,„^ 

....... .Pa.,.„. ,„,^„^„„^^ 
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■he con.ro. p.h Tnc of ...c -ncrc.cn.M forwarding inforn,.ion (hcrcnf.cr referred .o ns IF., should "fee 

eq.,.l ,o ,hc number of r.u,u3.ly e.c.u.ive increrncn,.! pieces in .he forwarding informMion. 

4. Ncxr Block Addre.. K.e.d: -^U Held is .he nex. block oddres. ,o lookup nf.cr .he cu.en. one. T^e leaf node 
insirvciinn ignofc^ (his field. 

5. S.Mis.ie. Offset F.eld: In d..3 switches, keeping Row s.a.is.ic. is as crucial as .he swi.ching da.o i,„lf. 
Wi,hou. keeping How Mn.is.ics i. would he difneu... M bcs,. .o mnnage , swi.ch. Having .his s.a.is.ics offsc. 
Held .llo-.v. one .o upda.e Mn.is.ic. n. various poin.s of ,hc parse. On an IP Router, for example, one could 
collec, p.->cke. coun. on v.-uiou. groups of DA. various Groups ofSA. all ToS. various protocols c.c. In another 
example dealing with an ATM .wi.ch. .his Held could allow ,he user to count cells on individual VPI or VQ or 
comb.na.,ons thereof. If. he designer wants to maintain 2' counters, then the size of.his Held should be s. 

6. r^T Off.t F.e,d: 71,i. i. an offset into the Forwarding Lookup Table. Fig. , 8. l...er discussed in more 
de.ai.. -n.c Forward.ng L.,ok..p Tnble has al, .he mu.ually exclusive pieces of informa.ion that is required .o 
build the final forwarding inrnrmaiion packet. 

Reference Hnrdwnre Design Example 

1T>c following i. an exai.iple of a h.irdware reference design for the parser u.seful with practice of .he 
present ir^vention. The reference dc.ign parser has .storage that contains .he packet/cell under scrutiny. TT,is storage 

Clemen, for the cell/fr.ame header inf. a.ion is to be .wo levels in depth. Tl.is creates a .wo-stage pipeline for 

header information into .he destination lookup stage of the Ingress Forw.vding Engine. TT,is is necessary because .he 
Ingress Forw.ard.ng Engine wiM n.,t he able .o perform a lookup un.il .he entire header information has been stored 
due .o the nexib.e starting point capability. Tl.e .wo stage pipeline allows ,he Ingress Forwarding Engine .o perform 
= lookup on .he present header information and also s.ores the ne. header information in parallel. When the present 
header lookup is completed, .hen .he next header lookup can proceed immediately. 

-n.= s.orage elemen. store a programmable amount of the incoming bi, s.ream. As on example, .he 
conngura,ion may be 6. by.es for IP datagrams and 5 bytes for cells. For an in.erface .ha. handles both cells and 
frames, the maximum of these two values may be used. 

A DMA Transfer Done signal from each DMA channel wi„ indicate .o a s.ntc machine .ha, i, can begin 
snooping and storing header information from ,l,e Ingress DMA bus. A packct/ce.l signal wi„ indicate ,hat .h= 
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header ,o be M.rcd is c.hcr 3 pnckc. hcodcr or a cell header When hcnder .nform.uon ha. be. . , 

orm.iuon has been completely stored 

from n DMA channel, a request lookup « ill be asserted. 

For Header .oo..,ps. tUere u il, be a registcr-based table which .,|, i.,;,,., 
Engine the l.o.up start-n^ pom. in the ,P Header Table. The ,r,,ress Porwardin, Engine uses the source interface 
number to inde. th.s table, this informatton alio, s the ,n,rcss Por.ardin, Hn,ine .0 s.art the se.cH at an, HeM In 
.he ,P header or Helds contained in the data portion of the packet. T^is capability, alon, wi.H the skip fu, 



^ill allow the Ingress roru.-„ding Engine to se.irch any fields . 
;cs per inierface.. 



; later 

ig them together to form complex 



A su,...b,c h,.rdware lookup .s sho.n in P.g. ,9 using a Parse Tree Graph lookup algorithm .0 determine a 
ror.3rd,ng..,s,on....isa,gor.„.m ^i-^ ^ n.bble or a byte at a t.me of either an ,P destination .ddress or 

VPl/VCl header. TH,s capabi,„v i. p.,.,..,,„ ^y software. Each lookup can have a unique tree structure .Hich 
.s po.n,ed to by one of si.een orig.na.ing nodes, one per interface. Tl.e or.ginating r,odes arc .stored in a 
programmable rcg.stcr-based table, allowing software to build these trees anywhere in the memorv . 



o the second stage Forwarding 



Mate maehmc controls the lookup process by examining the .status Hag bits associated with each lookup. These status 
a branch node. If it was an end node, then the lookup result is ,l,c indc.. 

i.dic.« ..ich ,i„ ,„ „„ ^.^^ ^^^^ ^^^^ ^^^^^^^^ 

clock enables and mux controls in activnte 



r...o.™.n„. ,„u „. ,„„„.^„_ ^^^^^^ ^^^^^^^^ 

m r,.„ ,.,„ ,„ „„, ^^^^^^^^^^ ^^^^^ ^^^^ 
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Queue M^nngcr wi„ need .o InnV. n. Ka for queuing of mu.ncnM pocke... ,o n,ul,iplc interfaces. For unicas, 
r.c.e.s. ro. e„.p,e. h,K of .He Po...ding ,0 can indic.e .He acina.lon in.crface „u..er and .He ..ainin, 
.6 bi.. .in p,ovide a Ln,c. 2 :D. T^c Layer 2 ID wi„ be used by .he Egress Forwarding logic ,o de.er.ine what 
Layer 2 header need, .o he rrepcndcd ,o ,he packe, da.a. For pacU.s. .hesc headers wi„ be added .o the paclce. as i. 
is moved fror. ,hc Egre. DMA FIFO (Hrs. in. Hrs, ou.) .o .he Eere.v, Buffer Memory. For cC.s. .he Layer 2 ID wi„ 
provide .he transmit device with the appropriate Channel ID. 

For unieau tr..fnc. the DcMinntion l/F number indicates the net.vork destination interface and the Layer 2 
ID indicate, .ha. type of Layer 1 header need, to added on.o the packe, da.a. For multicast, the multicast ID 
ind.en.cs both .he .ype of L..yer 2 header addition and which network in.erfaces can .ransmit .he multicast. T^e 
Egress Queue Manager wi„ pcrfo,^ a Multicast ID .able lookup ,o determine on which interfaces the packet wil, ge, 
transmitted on and w|,a, kir,d of Lnyer 2 header is put back on the packet data. 

An Example of Life of a P.nckcl Under the Forwarding Engine 

I. is now In order to expL.in examples of a simple and a complex packet through the Forwarding Engine of 
■he invention. On powc, .,p. r,g ,9. al, 2" blocks of the parse gr.aph are Hlled with leaf elements pointing .o an FLT 
offset that win eventually forw..rd nl. packet, ,o ,he Con.rol Proce^.or on the Ne.work Card. This is a dcfaul. route 
of all unrecognised p..cke... .Software i. responsible of setting up the default route. n,e way in which .he various 
elemer.,s a.e updated into „.i, p,.,e graph memory wil, he expl.ained for the .llt.s.ratcd cases of a simple multicast IP 
""^'^ 255.255.0 ^and a complex Hltcr packet, aging the 

simple IP Packet. 
Simple Multicast Packet 

On power up. .he entire blocks in ,he Parse Graph Memory may be assumed .o be filed wi.h leaf elements 
.hat poin. to r off..e. of n.T .l.ich wil, route the packe.'.o ,he Network Proccsor. Le. i, now be assumed for .his 
"ample, .hat the ing,e.,s packet h.a.. n destination IP Address of 224.5.6.7. In this case. ,he hardware will lookup .he 
224-^ off..et in the block ...e f.r..t lookup block is nUo called orieinnting node) and Hnd a leaf. -n,e h.dwa.e wil, 
cr,d the scvch and look up .he default offset in the 224* location and look up .he FLT and forward .he paeke. ,o ,hc 
control processor. 
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Ihc snfiware Tirsl h 
llic snfivv.-, 



When .h= ccpTol processor forward. ..uhscqucm pnckcs of DcMin.tion IP cddrc.s 224.5,6,7. it will 
gcncrni- ilic gr.iph shown in Fif. 21, 

Mc the parse graph locally. The parse graph created is li.Mcd a.. 1-129.2-131. 
look., up the nrs, block n,k,n oricinatine r,odc. 77,= ofrsc, in the Hr.M block is 224. which is the 
nr.. b, ,c of ,„e de.Mina.ion IP header. 1. f.nd. a defaul. route. - an indication for software to allocate a new block for 
all suK.equen, bytes of the des.inn.ion IP address. Once the sof.warc hits a default route, i, knows tha. this is , link 
node, r„.,n ,he l.nk node onwards. ...c .sof.wnre has to alloca.e new blocks for every byte it wants the h.dware ,o 
search for a .a.ched destina.lon IP address. Through an appropria.e sof.ware algorithm, i, f.nds tha, 129. 2. 131 arc 
the nex. .nrcc avail.nble blocks to use. The software will .hen install eon.inua.ion element with BA of 2 in the 5* 
offse.orblock,:9.con.lnua„one,e,nen.wi,bBAofl31in6^offse. of block 2. and a leaf e.err,en. of FLToffse. 5 
n. 7'' of r.c, of block 131, Once such a brand, wi.l, a leaf is created, .he node link is then in.stalled. The node has to 
he installed las. .n the new leafed branch. The node in this ca..e. is a continuation ele.nen. with BA of 131 a, offse. 
224 of il.c I" block. 

line hardware is now ready fn, any subsequent packets with destination IP address 224.5.6.7. even .hough i, 
knows nothing about ,.. Now . when ,!,e h.^dware sees the 224 of the the destination IP address, it goes to the 224^ 
offset of I " block of the parse graph and Hnds a continuation element with B A of . 29. Ihe hardware will ,hen go ,o 
.he 5- offset (second byte of destination IP address> of the 1 29'^ block and f.nd another eon.inua.ion elentcn. with 
BA of 2 TTe hardware will then go to f,'^ off.ct (third byte of destination IP address) of the 2^ block and find 
another c.r.nnuation element with HA of 1.3 I. n„ hardware will ,hen go .o 7'" offset (fourth byte of destination IP 
address, of the , 3 1 - Nock and find a leaf ele.nen, with FLT of 3. -Pte hardware now know.s .ha, i, has completed .he 
ir match and wil, forward the forw.arding ID in location 2 to the subsequent hardware block, calling .he end of 
packet parsing. 

I. ..hould he noted that .he hardware is simply n .slave of the parse graph pu. in memory by software. TTtc 
Icng.h of the search purely depends on the software requirements of parsing length and memory si.e. TTtc .dvcr,c 
effects of such par.sing nre .si.c of mcmorv. and search time which is directly proportional ,o the Icng.b of the search. 

In -his ca.se. the .search will result in the hardware effecting 4 lookups in Parse Graph and I lookup in FLT. 
P.ickct x\lih Mask 255.255.255.f1 
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Building upon n the p.nr.c grnph in Fig. 20. a packet wi.h an illuMralive mn.sk 255.255.255.0 and address of 
4.6.7.» is now insiollcd. In ihis cnsc. the sofiwarc will go to the 4'* offset in the originating node and find a 
contintjation clcmcni xvi.h DA of 129. The .software will (hen go to offset 6 in block 129 and Tind a default FIX 
offset. The sofiwarc then knows that this is a link node. From now on. it has to allocate more blocks in the parse 
graph, such as block 2. At off..ct 7 nf block 2. it will install a leaf element with FLT 3. TTicn it will install the link 
node consisting of writing a continuation element with BA of 2 at off.-iet 6 of block 129. 

When the hardware receives any packet with the header 4.6.7.x. it will look ir,io the 4'" offset originating 
node and find a coniinuation clement with BA of 129. then look at the 6'" offset in block 129 and find a continuation 
element with D A 1 .H. and then look at the leaf element at offset 7 with FI.T of 3. Tliis FLT will be of value 3 which 
is then forwarded to the Buffer M.nnaccr and eventually the Egress bandwidth manager. 
Packet with M.isk 255.255.0.0 

This si.b.seci.on will huild fpon the parse graph in Fig. 20 and inMnll a p.ickct with an illustrative mask 
255.255.0.0 and addrcs. of 4.R.x.y. In this case, the software will go to the 4- offset in the originating node and Find 
a continuation elemcni wi.l, BA of 1 29. T>.e software will then go to offset 8 in block 1 29 and Fmd a default FLT 
offset. At this time the snftw.ire know, that it has to install a new FLT (say 4)offset in the offset of block 129. 

Tl.c hatdw.ire when receives nny packet wi.h the header 4.R.x.y ii will look into ll.c 4'*' offset originating 
node and fmd a c.niinun.ion elemcr, with B A of 1 29 . then look at the leaf element of block with FLT of 4. and 
terminate the search. In this cn-;e the h.nrdwarc will do only 2 lookups. 
Complex Filtered Packet 



Now assume that there was a requirement to filler a packet with header 4.5.6.8.9.x.y.z. 1 1. Ther 



: are no 



Testr.ctions to the above concept of parsing the packet, and the time it takes to parse ihc packet will increase since 
the hardware will have to read and compare 9 by.es. TTte hardware will ..imply keep parsing however until il sees a 
leaf element. The x.y .r bytes are blacks which contain continuation elements pointing to the next block with all 
continuation elements of x pointing to block y. all continuation elcment.s of y pointing to block z, and all . 
continuation elements of 7 pointing to the block which has entry 1 1 as a leaf, and .he rest being default. This is where 
the fork element comes inin play and may be called up to lookup the forwarding at the end of search 4.5.6.8. 
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Removing Simple IP Miiliicn.M Packets 

77,0 r=,.ovo, of p.ckcK M.n,l3r .o .h= reverse of ndding nddrc, .o ,hc parse grnph. above explained. TT^c 
p";ucdncodc for removal in thi'; cmlindimcni is as follows: 

W:ilk down lo end nf leaf rcincn.hering cnch block address nnd off^t in block. 
FOR ( Prcim Lc.nf nndc lo nriginniinc node) 
in ( only element in block) 

^ci dcfauh FLT offset at the previous NODE offset address 
free the last block 
goto previous block 

ELSE 

SCI default FLT offset .nt last Icnf. 

HNDIF 
END rOR 

Egress Bandwidth Manager 

Ever>. VO Module con.cc^ a NeoN pon to one or multiple physical ports. Each I/O Module supports 
multiple trafHc priorities injected vi.n a single physical NeoN Port. Each traffic priority is assigned some bandwidth 
by 3 network rr^anager. as illustrated in Fig. M. being labeled as the "QoS (Packet & Cell)-. 11,= purpose of this 
section is to def.ne how bandv^idih is managed on multiple Iraff.c proFtlcs. 
NeoN Queuing Concept.-; 

The goal of NeoN Quc.i.ng. of the invention, thus, is to be able lo associate a fixed configurable bandwidth 
with every priority queue and also f, ensure maximum line utilization. Traditionally, bandwidth enforcement is done 
in systems by allocating a fixed number of buffers per priority queue. This means Ihai the cnqueinf of data on (he 
priority queues enforces bandwidth allocation. When bMffers of n certain queue are Hlled. then data for .ha, queue is 
dropped (by not enqueuing data on tl.a, queue), this being a rough appro.xima.ion of the ideal requirement. 

■n,ere arc many real life analogies to understanding the concept of QoS of ihe present irtvention. e.g. cars on 
.-. highway u ith multiple entry rn.nps or moving objects on a multi-chnnnelcd conveyor in a manurac..lrir,g operation. 
For our purposes. le, us examine the si.nple case of "cars on « highu-ay". Assume .ha, 8 ramps were to merge into 
one lane a, sn,„e pmnt on the hiph^ay In real life experiences, everyone knows thn, this could create traff.c jams. 
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Bu. if ..„3,cd cccC, ,hc ngh, QoS). .hen .he sln„c highw,, ,.nc c.n be u.i.i.cd for 

...ho.. T.is .cons .h. .he. is no dis„nc.ion He.wccn an o.bu.ance on one .amp and someone headed .o .he be.h 

on ano,hc: ramp. Bu, in .he me.hodology of .he inven.ion. we deHne ccnain preferen.ial charac.eris.ics for certain 

en.,, ..PS T.=. a. diffe.en, mechan.sms .ha, .c can crea.c. One is .o send one ca. f.om each cn.r. ra.p i„ , 

round rohin fash.on. i.e. each ramp is e,ua.. T>,is r.eans counting cars. B.. if one of these ••cars" turns ou. to be . 

tracor .r.iler with 3 .rai.ers. .hen in f.c. e,ua> service is no, being given to aU en.r, ramps as measured by the 

amoun. of highway occupied. ,n fac. if one entry ramp is all .rac.or .rai.ers. ,hen ,he backup on ,he o.her ramps could 

be very signiHcan.. So it is impor.an, ,o measure the size of the vehicle and its importance. TT^c purpose of .he 

• uafnc cop- (a.a QoS manager) is ,o manage which vehicle has the righ. of way. based on si«. imponance and 

perhaps lane number. T^e -.rafnc cop" can. in fact, have different instructions every o.her day on .he lane entry 

charac.er,s„cs ba.ed on wha, .he -.own hall manager" a.a ne.wor. manager has decided. To conclude the concept 

of QoS understanding. QoS is a mechanism uhich allnw. rrr„;„ A , 

m uh.ch allows ccr.am da.aprams.o p.iss through queues in a controlled 

TTianner. so as .o achieve a deterministic and desired goal, which may vary from applica.ion to application e.g. 

bandwidth uuli.a,ion. precision b.nndJld.h allocation, low la.ency. low delay, priority etc. 

T>e NeoN Queuing of the inven.ion handles the problem directly. Neon Queuing views .he buffer 
allocation a. an or.hogonal parameter ,o the Queuing and bandwidth issue. NcoN Queuing wil. .i.erally segmen. .he 
physical w.re into small time uni.s called -Time Slice" fas an example, approximately 200 nanoseconds on OC48 - 
.in,e of M byte packet on an OC.8). Packets f,om ,he back-planc are put into the Priori.y Queues. Each .ime . 
pncke. is ex.rac.ed from a queue, a .imestamp is also tacked along wi.h tha, queue. TT.e time ..amp indica.es disunce 
in time from a -Curxen. Time Counter" in Time Slice Uni.s. and when ,he next packet should be de-queued. TT,e 
•distance in time' is function of a, packet si« information coming in from the back plane, b) the si.e of Slice Tl.e 
i.«>r and c, the bandwidth a.l.Ka.ed for the priority queue. Once a packet i.s de-queucd. another counter is updated 
v^hich represents the Ne.x. Time to De-queue (NTH.) - .such purelv a function of .he size of the packe. Jus. de- 
...uedNTTB is one for ce,,.based cards. because3l,packet.,are the samesi.eandr.t in one Huff. 

Provestha. .he NCON EgrcssBandwid.h Manager is monitoring.He,ine.odeterrr,ineexact,y.ha.next.osend.™^ 
-cchanisn,. therefore, is a bandwid.h manager rather .han jus, a dequeuing engine. 
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The NroN Qucuinr of tb- prcscnl inventinn mnr^r,^ ^ ^ 

on. morco^c^. may he .hough, of a. TDM scheme for allocating 
bo.d.id,h ro. d.rferen, p.on„c. u.n, p.ori.y queuing for ABR (Ava.UMc Bi, R.e, b.ndwid.h. Added 
.dvan,ages of ,he NeoN Queuing .e ,hn.. wi.hin .he TDM mech.ni... hand.id.h i. ca.cu.a.ed no. on pac.e. 
coun.- hu, on ,.c.e. H,,e M.e". 1 hi, g.nu.oH., is . m.ch hc.e. .ep.ica of .he ac.un, hand.id.h .ih^.^on and 
.no., .3nd.id.h cn,cu,..on, ..he. .han si,.u,a.ed.p,o..a.lon. ... .cond 'NcoN Advan.a.e- i. .Ha. .he 
Ne.wor. Manager can dynn^icn-.y change .he bandwid.h requircmcn,. si.nilarlv .o a sliding scale on a volume 
con..l. T.i, i, fea.H,c .i.cc ,hc Hand.id.h calcula.ion, fo. p.o.., ,..eue, ..c no. a. al, ba.ed on huff, allocations 
In Neon Qucu,ng. ..he. .he hand.id.h alloca.ion is ba.d on .he .i.e s.,cing .he bnnd.id.h on .he physica. wi.. 

This type of bandwidih mnnagerrcni is absolutely necessary when running „ u- u ,- 

y ecessary when running at very high line speeds. .0 keep line 

utilization high 

Mathematics Used during Queuing 

First we u-,11 develop .he variables and cons.an.s being used in .he ultimate mathematics. 



Symbols 




TS 


-Time Slice of bandwidth on wire used for calculations. (200nSec 
for OC48). 


NTTS 


Ne.. Time To Send. 1 his number m un.ts of Ts representing a 
address to de-queue from cuncnt lime. 


BitTimc 


Time penod of a s.ngle b„ on .he w.re ol ,he current I/O module 




Delay factor ,n Number of TS. reprccn.ing bondwid.h ' 
calculations set by Network Manager, for priority Queue n. 


BWn 


Bandw.d.h of Queue n m Perccn.agc as entered or calcula.eT^ 
the CPU Sofiware. 


Pn 


Number of Priority Queue.,. ' 


TBW 


Total Bandwidth of .he wire ' 


NTTD 


Nexilimc To Dequeue. 


CT 


Cunent Time in TS units. 
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Consider r..M .he u.cr in.c.f.cc levc, ,o sec ho. K.ndwid.h is nl.oc.cd a.ongM vonous pnori.i„. .he user 

■ s non..,,, .he ,oH of divid.n, ,00... K.nd.id.h ..ongs. vn.ous p..ori..s. T>e user cou.d a.so be p.scn.cd 

u.,h .,...,n, .p ,he en.i. Hond.id.h in bi.s pe. second (ns . c..p,e for 0C.8. i. .^d be 2..Gbi.s,. ,„ ei.He. 

C3se. son,c CPU sof,.nre cnlcu.n.es n number pnir. prioH..v.^n. .on, ^.pHori.y or mBi.s/secpnori.,. Since .he CPU 

h doin, .His cMc..n,ion. i. can be ensi., cH^n.ed b.sed on .he .0 .odu.c. T.e Bandwid.H Manager does no. need .o 

knnw nhou. .he I/O module .ypc. only earing about .he priori.y-An mir Thus if , 

I ^"P-^'f- Ibusifauscrconnec.cd.oihe NcoNport 

Ihn. canno. hnndic dnin a. full line rate, the CPU can chanee this v,l„, .r, .a- , r ^ 

cnange ih.s value lo adjust for .he cus.omer requiremcn.s. 

An= lOO/BWn 

NTTS. = ((Packet Byte Count • Bi.T.me) / fTS)) • A„) . NTTS, , 

.OO'^ or 3 TS .imc. as .,.e approaches inHnity. hV^:' in^^ ^ra^^r" y^^^^^^ '° 
Next Time To Dc qocuc is .he .Im- .hit u.7- ci-,rr ih- a 

„ p™.,„ .... „ „„ „;:-; :^.s^rir/;s:; ^^.^a"' 

NTTD. = ((Packet Byte Count • BilTime) mod(TS)) + CT 
Queuing Processing 

I. r,ow in order .o decide the processing needed .o queue addresses from .he back-plane on .o ,he Priori.y 
Queues, r,p 24. .hich depic.s the overall queuing and scheduling process. Con.rol Da.a. which includes da.agram 

addresses, from the 'NeoN Control Dita ^wiirh- ic a - . 

control Data S«,.ch . ,s sor.ed .n.o pr.or.ty queues based on .he QoS inforrr.a.ion 

embedded in .he con.rol Data, by the Queue Enpinr Th, ^-^ ^ 1- n ■ 

Dy yueue Engine. TT,e Scheduhng Engme operation i.. rendered independent of 

.he Queue Engine which schedules dn.agrnrr, addresses through use of the novel algorithms of the invention listed 
further below. 

Tlie queuing Engine has the following lasks: 

«p.ority;ueues:^rr::c^^^;jc:;r:n;:::^,:^^^^^^ 

-..crmatlsse.for'Iqur' ^^"^"'"^ P^"- '"gress based on 

Drop Packets .S.ar. [dropping pnckcs when .he Priori.y Queues arc full. 
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Tor c.ch Priority Queue P.. .here will be n -head pointer - pHend." nr,d a '.ail pointer - pTnil/. Input Fifo 
feeds the priority Queue?; P. witl. buffer nddress from the bnck-pbnc. Additionnlly. there is a forward. For OC48 



!. and 3S<;uming 64 hyic poikct^ as average size packets, the folln\ 



ng processing will be done in about 



200nSccs The prefencd p.ieudo code of the invention for the En-queue Processor is as follows; 

Rc.nd input Fife. 

Find priotiiv o) ilic p.ickcl 

IRrnom on queue) 

move buffer frdm Input Fifo to *pTail, prinritv queue 

Advance pTail„. 

increnicni buffer count on queue 
IFfpackcl count on >= watermark of that queue) 
set h.ick-pressure for that priority 

r.NDIP 

ELSE 

move l>uffcr from liipul Fifo to drop queue 
Up.f.iic M.nl.Mic.^ 

EN DIP 

The verb.M e Tpl.nn.ition of the psucdocode listed above. As each control packet is read from the 'Neon 
Control Da,.-, Switch' is put onto one of N queues after it is veriHed for physical space available on the queue. If 
there is no mom set on the queue the d.-,ta is put on a drop queue, which allows the hardware to return addresses back 
to the origln.-,.ing port vi.-, the NeoN Control D.Ma Switch'. AI..0 n w.^ermark is set. per queue, to indicate to the 
ingress to filter out non ptcfcrred traffic. TTiis algorithm is simple but needs to be executed in one TS. 
Scheduling Procc.';.';ing 

This section will list the algnrithm used to dc-qucuc address frtim Priority Quct 
This calculation also has 10 he done during one TS. 



:s Pn onto the output fifo. 



Wait here till CT == NTTD AND no back pressure from output fifo. // sync up 

rr.'n^AVn^ .. .. // somc variable. 

FOR (all P. . High lo Low) 

IF (pMcad, != pTail,) 

IF(CT>= NTTS.) 

Dc-Qucuc (pHcad.) 

Calculate new NTTS. // see cquaiioji (2) above. 

Calculate KTTD // equntinn (3) above. 

update statistics 

X=TKUE 

ENDFOR 

ENDIF 
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ENDir 
ENDFOR 

IF (X== FALSE) 

FOR fall P. .High to Low) 

IF(pHcad, !=pTail,) 

DcQueuc (pTail.) 
update sialislics 
X=TRUE 
ENDFOR 

ENDIF 
ENDFOR 

ENDIF 

IF(X=FALSE) 

ENDIF 
Update CT 



n Dc-Qut 



Is conceptually a simple routine, listed below.- 



Dc-Oueuc(Q„) 

•pOuipuiyTniU* = •pHcad.+4 

explanation of the p,uedocodc l.sted above is that there arc two FOR loops in the algorithm - .he firs. 
FOR loop enforcing .he co.nn,ittcd bandwidth to the <,ueue. and the second FOR loop serving for bandwidth 
utiliration. sometimes called aggregate bandwidth FOR Loop. 

Examining firs, the Committed FOR Loop, the queues are checked from the Highest Prior.ty Queue .o the 
Lowes, Priority Queue for available datagram to ..hedule. If a queue h.s available da.agrarr,. .he algorithm will 
, check to ,.ce if the Queues Time h... tn dequeue, by comparing its NTTS„ agalns, CT. If .he NTrS„ has fallen behind 
CT. .hen the queue is Dequeued: otHerwi.e. the search goes on for .he nex. Queue until all queues are cheeked. If a 
data from a queue ,s scheduled to gn out. a new NTTS. is calculated for .ha. queue and a NTTD is always calculated 
_ When any queue is de-queued. When a Network manager assigns weigh, for .he queues, .he sum of all weights should 

-be ,OO..S,nceNXrS.isba.sed on datagramsi..e..h=ou.pu, data p. ,.cue is. vcr. accurate imp.^^^^^^^^ 
the bandwidth sc. by .he manager. 

Let us now cxatnine .he Aggregate FOR Loop. T^is loop is only executed when no queue is de-queued 
during .he Committed FOR loop. ,n other words only one de-queue operation is performed in one IS. .n .his FOR 
Loop, all queues are checked from Highest Priority .o Lowes. Priority for available da.a .o dequeue. T^e algorithm 
.0. in .his FOR Loop for one of ,wo reasons: cither .here was no da.a in all .he queue., or .he ^TS. ofoM queues 
were .Mill ahead of CT fit was no, ,ime to send,. ,f .he algorithm entered .he aggregate FOR Loop for empty queues 
around .he fate wi„ he ,be same. However if .he aggrega,e FOR Loop was en,ered because .he 



then ihe $( 



-3.1. 



wo 99/35577 



PCT/IB98/01940 



mrS. no, rcnchcd for nil queue, .hen .he nggrcgn.e w|,l r.nd ,hc highest priorhy such queue ,nd dequeue >,. 
also in ifi.ii cnsc il would upjnic NTTS. and calculate NTTD. 

■n,e algorithm hn. built in credits for queue that do no, hnvc data to de-queuc in their .i,T,c slat: and dcbi.s 
for dntn ,hn, i, dequeued in the Aggregate Loop. -r>,ese credits and debits can accumulate over large periods of .ir.c. 
■n,e debit and credit accumulation time is a direct function of the si..e of NTTS^ Held in bits, for cxarT,pl= a 32 bJ, 
number xvould yield 6 minutes in each direction at using I60nSec as TS (2^ - I60nSec). Each individual queue 
could be conHgured to loo.c credits and/or debits, depending on the application this algorithm is used. For example 
if the algorithm . as to be u.ed mainly for CBR type circuits one .ould wan. to clear ,he debits fairly quickly, where 
as for bursty trafHc ,hey could be cleared rather slowly. Tl,e mechanism for clearing debits/credits is very simple, 
asynchronously setting NTTS. to CT. If NTTS. is w.y ahead of CT. Queue has build a lo, of debi.. .ben setting .h= 
NTTS. to CT would mean loosing all the debit. SlmiLnrly if NTTS. bad fallen behind CT. Queue has build a lo. of 
Credit, then setting NTTS„ lo d would mean losing all .he credit. 

Example of Implc.nenting CBR Queue Using the Algorithm 

It is now appropriate to examine bow ,o build a CBR queue out of the algorithm listed above, again 
referencing F.g. 2-1. Le, it be assun,cd that the output wire is running n, OC48 .speeds (2.4Gbits Per second) and .ha, 
Queue 1 (h.ghest Priority Queue) h.s been assigned to be the CBR Queue. Tl,e way we conHgure ,he weigh, on ,h= 
CBR queue is conHgured by su.nm.ng all .he input CBR How b..ndwidth requirements. For.ake of simplicity .here 
are 100 Hows going through .be CBR Queue, e.nch wi.h a bandwidth requirement of 2.4 Mbi.s per second. T>,= CBR 
Queue bandwidth will ,hen be 2.4Mbi,s/sec Times 100. i.e. 240Mbi.s per second fi.e. 10%). In other words, 

QRATE,^ = Z Ingress Flow Bandwidth. 

A, = 100/10= 10. Based on Equation 1 

NTTS. would result in 10 every time a 45 bytc datagram is dcquucd. - Based on Equation 2. 
. rmS. would resul, in 20 every time a 90 byte datagram i.s dequued. - Based on Equation 2. 

hOTD would result in I every time a 4.-; byte dat.tgram is dequeued. - Based on Equation 3. 

NTTD would result in 2 every time a 90 hyte datagrnm i.s dequeued. - Based on Equation 3. * 

This shows that the queue will be dequeued very timely, based on datagrann si« and the 9c of bandwidth 
allocated to the queue. TT,is algorithm is independent of wire speed, making i, very scalable, and can achieve very 
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high d.,a speeds. This 3,ogori.hn, also ,akcs do.agrnn size in.o accoun. duHng schcduting rcg.dlcss of a .h= 
daugra. being a cell or o pacVe,. So long as ,hc network Manager se.s .he weigh, of .he queue as the sum of ,11 
ingress CBR How bandwidth, .he algorithm provides .he scheduling very accura.ely. 
Example of Implementing UBR Queue Using the Algorithm. 

1. is ver^ simple .o implement a UBR queue using thi. algorithm. UBR standing fo. .he queue 
which uses the left over bandwidth on ,he wire. To implement .his type of queue, one of N queues with 0% 
Bandwidth, and .hen this queue is dequeued when .here is literally no other queue .o de-queue. T^e NTTS wi„ be 
sc. so far in the future .h.n, af.cr ,he algori.hm dequeues one da.agrom .he nex, one is never scheduled. 
QoS Conclusion 

AS has been demonstrated, the algorithm of .he invention is very precise in delivering bandwidth, .nd its 
granularity is based on the si.c <.f TS being independent of Ccll/Pac.et information, and also provides all of the 
a™ services required: implying not only packet., also enjoy the ATM services hu. cells and packets coexist on the 
same intcrf.ncc. 

Real Life Network Manager Examples 

TT,is section will consider different Network Managemcn. bandwidth management scenarios.all well 

handled by .he inven.ion. .nso far as ,he NeoN Network controller i. concerned, .here arc „ queues egress (as an 

example i. could be R). each queue being assigned a bandwidth. T1,e Egress Dandwid.h Manager wi„ deliver .ha. 

percentage very precisely. T.e Network Manager can also decide not .o as..gn ,00% of .he bandwidth .o all queues 

in wbieh case .he .eft over bandwidth will simply be distributed on a high .o ,ow priority basis. Beside these .wo 

levels ofcnntrol. the Network Mnnarcr can also rxnmln^ ,- .• 

Manager can also exammc statistics per priority and make strategic statistical 

decisions on it own and change percentage allocations. 
Exemplary Ca.-Jc 1 : Fixed BancJwidth 

.n .his scenario. ,00.. of .he bandwidth is divided into all queues. ,f all queues arc full a. all ,i.es ,hen the 
queues will behave exac.lv „ke Pair Weighted Queuing. T.e reason for this is .hat - the Egress Bandwidth Manager 
w... deliver .he percentage of the line bandwidth as requested by .he Network Manager, and since .he queue, are 
never empty. .he egress bandwidth does no. have .ime .o execute .he second FOR loop (Aggregate Loop), above 
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rr .h= ,...s nrc no. r.l, 31. ,hc .i.e. however, .hen durin, .he .I.e ...e queue is e.p.y see o.her ,ucue 
may be serviced abend of i.s .ime wi.hou. a charge agains. i.s bandwid.h. 

A. an example,,he Nc.work Manager decided .o alloca.e I 2.5 % bandwid.h .o every one of ,he eigh, 
queues. ,hen ,he Ne.work Manager has ,<, provide ,o .he Egress bandwid.h Manager.- 
A,-Priori.y List of all A. one for each priorily. 

Bi. TirT,c Based on I/O Module Egress Bandwidth Manager is running on. 

For a band.id,h of ,2.5 ... would calculate to be 8.00 ( 100/1 2.5). For a OC48 Bi. Time would calculate 
.0 be 402 pscc. 

Exemplary Case 2: Mixed Bandwidth 

.n .h,s example, not all of the bandwid.h is divided into all of the queues. ,n fac. .he sur. of al, f.ed 
bandwidth on the queues i, no. 100. of .he bandwi.h ava.lable. The Egress bandwid.h Manager wil, deliver .he 



bandwidth on ,hc queues up .o .he al.oeated amount, and .hen aggregate .rafHc amongs, .he priori.ies on .he 
bandwidth. T.is guaran,ees so.e percen. of a class of .rafHc .o t.a.e i, .hrough ,he pot. and also provides 

pHori.i«d trafnc. Por queues .ha. are no. full durtng , he allocated .i.e. .hat bandwid.h will be , OS. . ^ 
bandwidth. 

Exemplary Case 3: No Mixed Dandv^-jdih For All Queues 

• n thU scenario,07, is allocated ns Hxed bandwidth for all queues. The queues will .hen behave purely like 
pHori,i«d queuing. The Hrs. For Loop l.sted in section 0 Scheduling . wi„ considered as NOP. 
Exemplary Ca.se 4: Dynamic Bandwidth 

In this illustration, the Ne.work Manager may initiallv com, kj k,- 

gcr may initially come up wi.h No Mixed Bandwidth for all Queues 
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r, SHO.M obsccU .h. ,„c ..Hove c.cs .n.p.s on.,. ,Hc nppHca.on or.Hc a„o.. 

F.r.hcr .ndincn.inn, occur .o .hose ski.cd in .hi. .nd such nrc considered .c 
the spirii ond scope of ihc invcn.nn defined in ,hc appended claim.v 
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Clai 



.^A ,„„,,„^ ^^^^^ ^^^^ ^^^^^^ ^ ^^^^^^ ^^^^^^^^ 

.p.c„ „ . d» „,..o,«„, „„ .^^„,„^ ^^^^^ ^^^^ ^^^^^^ ^ 

t both cell and packet dala 

forwarding w.thoul imp.iclinp ihc correct forwarding characteristics of either. 

' " " - ' ' ' p.-... . . „„„„„ 

forvvard.ng enpinc w,ih common algorithms independent of context-sensitive inform-,,- 

context sensttive information contained in the cell or packet. 

from the forwarding engine is passed to a network 
rfovidc minimum cell 



-t.A method as claimed in cl.->lm 2 wherein the inform 



nnger and thence to a network egress transmit facility and in n manner such a 



..A .c.od as Claimed inc,aim3wherein,uali.,ofserviccinformationi.inc.udedintheinfo^a.ion pass. 

r...Hcforw.ard,ngeng,neandm..naccdh>..he,.uemanagerfor.o.hcc,,sand packets sirnultane^ 
the common algorithm. 

5. A method ns claimed in claim 4 wherein a common parsing algorithm is also u.sed for similarly forwarding both 
cell data and data packets. 

fi-A method ns chimed in rliim .u 

.h= ^,„„,,.^^ ^^^^^^^^ _^ ^^^^^^^ 

.^pace exisis on the queue. 

7. An.e,hodasclaimedinc,aim.wherein..houldthereHenosuch..pacc..hedataisputir,ndrop<,ueu^ 
returned hy the switch to the ingress of the network. 

8. A methrxJ as claimed in claim 7 wherein a watermark is set for 

a.ermark ,.s .set for each queue to instruct each ingress to Hlter out 

non-piefcricd data traffic. 

Hnsed upon time slicing the bandwidth 
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10, A mcthoJ .s cla^,cJ .n d.,m 9 whcrcm Ih. network m.na,..cr Jyn^m.o.lly v..r.<.s lUr bondwiJth 
requirement. 

n. A mclhoJ of proccssui,-, uifortujlion .oiUamcU in Jjla and Jala pa.ki-U r.-.viv,-J al Uk- in,T..ss ,.f a 
data nctworkij^i; system, that compr.s.^s. applyinj.. both tlv received data ccUs and data packets lo a common 
data ^or.^.a^dmc and roufnr, switch; mana>;i;,t; both cell and packet data switchin,- in the common switch u.ini; 
common hardwarr, common quality of service- algorithms, and common rorwardinj., alyorilhrns; and 
ControUinc the packet sw.lchini; ind.-pendently of and without interferini; with ihe . el! Jala swilchinj.,. 
12. A mctl,od of processin,; packets of i^fomialion from a forwardinj.. switch and queue mana,;in« the 
fonvardinc of the same, that comprises, as each pack.'l is read from the switch, putln,,-, the same into one of a 
plurality of queues after it ,s ver.fied that available physical space ex.sLs xn the queue; pl^cmy, Ih.. packet 
information in a drop queue should there be no such space and relurmn,; the packet niformal.on llirouch the 
switch- scttini; a watermark for each queue to enable the filterin,; of non-preferred informat.on ir^m,-. and 
aUocating for different pnont.es by packet byte s.z.- and based upon tune shcin,; the bandwidth. 

13. A system architecture app.,r,alus for simullaneouNly pn.cessm,; inforni.ition , onl.,in<-J in d..U. , ells and 
data packets received at the ioj-.ress of a data networking, system, .said apparatus havn,.... >n . ombination. m.-ans 
forapplyii^c both the received data cells and data pac kets from the in,..r.-ss lo a ccmn.on Jala switch within the 
system; means for cntroUini: U,e sw.lch for cell and packet indiscrimmately. for forwarJin,; by a common 
algonthii, based on control ii>forn,at,on contamod ii, ihe cell or packet and without transformin,-. packets ml., 
cells; and means for conlroll.nr. w.lh a common bandwidth mana,..emenl al,..orilhm bolh cell and packet data 
forwarJinc without impactin,-, the corrcit forwarJin,; charactcrLslics of either. 

14. Apparatus as claused ,n claun 13 wherein the c.rll and packet conlr..! informaluu, .s rroc.-ssed in a 
common forwardin,.. en,..u.e with conunon .l^.orithm.s. independent of conloxl..sen..iliv.. inf<.ro,ati<.n conlain.J 
in the cell or packet. 

15. Apparatus as da.med .n .laim 14 wherein means i.. pr..vided for parsin,; the u,forma.io„ from the 
forwordinr, cn,..me to a network e,..ress queue mana,.er and ihenc to a network ..,..ress transnut facility, and in a 
manner such as lo provide minimal cell/packel delay variation. 

16. Apparatus as da.med .n cla.n, 15 wherein quality of scrvi. e informal.., .s included in the ,nfom,al,o., 
passed from the forward.nj- en,.,ne and manaj;..d by Ihe qu.-ue manaj-.er for k.lh c.-IIs anJ packets 
simultaneously based upon the common jl|;orilhm. 

SUBSTITUTE SHEET (RULE 26) 
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17.APP3..S .3 cl..cd in c>.i. ,6 .Herein . co..on p...n, .„„H.h. is Mso provided for s.-.i..., 
forwarding bosh cells and daia packcls. 

physical space cxisis on the queue. 

IP.Apparniu^ as claimed in claim 1 8 wherein, should there he nr. crh 

there he no such space, rrieans .s provided for the data lo be 
put in drop queue and returned hy the switch to the ingress of the network. 

.O.Appara.usasc.ai.nedinc,aim ,9wWinawaterm..klssc.foreach queue to .st.ctsuchingress.o niter 
out non-prefcned data irafric. 

Il.Apparatus as claimed in claim 18 wherein meant it nrr, • i j r .. 

means ,s prov.ded for allocat.ng bandwidth for different priorities by 
packet byte size and based upon time slicing the bandwidth. 

22.Appara,us as claimed in claim 2 . wherein the network manager dynamically varies the bandwidth requirement 



interfaced. 



n 14 wherein, between the ingress and the .switch, a VCI function/ assembly is 



25. Apparatus as claimed in claim 24 wherein said assembly connects not only to the switch but also to a Header 

.ockupandfor..3rdmgenginef.rho,h.hecel,and packet d..a;withtbe engine conncctingthroughacontro. data switch 
and a quaUty of service managing module to a buffer, also inputting from the output of the switch. . 

26. Apparatus..c.imedinc,aim.5whereinthebufferfcedsacenda.aVCsbapingc.>cuit that connects with the 
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loul packat langih 



Source Address 



Destination Address 



SUBSTITUTE SHEET {RULE 26) 



PCT/1B98/01940 



Time-Division Multiplexing 



Q ED QD 
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Forwarding 
Englna 



Physical 
Layer 



< □ IZZI CD I 

KI ATM Inlarface with SAR 



Phy3ical 



^1^ 



Packel-only Interface 



ATM lnl«rtace with No SAH 
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F((. 5 PRIOR ART 
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^ mamory 
Data Is removed Irom mamory and v 
trmnimittad ' 



Memory 

- Buffers 

- RCV Queues 

- XMT Queues 



f 3 



r.». 6 PRIOR ART 
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Cross-bar Switch 



Data it recaived and pul into 

Data Is remov«d liom memory and ■ 
irantmitted 




Fig. 8 PRIOR ART 
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